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[i|  We  have  developed  a  technique  to  provide  short-term  warnings  of  solar  energetic  proton  (SEP)  events 
that  meet  or  exceed  the  Space  Weather  Prediction  Center  threshold  of  J  (>‘10  MeV)  =  10  pr  cm  2  s  1  sr  \ 

The  method  is  based  on  flare  location,  flare  size,  and  evidence  of  particle  acceleration/escape  as 
parameterized  by  flare  longitude,  time-integrated  soft  X-ray  intensity,  and  time-integrated  intensity  of  type 
111  radio  emission  at  ~1  MHz,  respectively.  In  this  technique,  warnings  are  issued  10  min  after  the 
maximum  of  >M2  s'oft  X-ray  flares.  For  the  solar  cycle  23  (1995-2005)  data  on  which  it  was  developed,  the 
method  has  a  probability  of  detection  of  63%  (47/75),  a  false  alarm  rate  of  42%  (34/81),  and  a  median 
warning  time  of  --55  min  for  the  19  events  successfully  predicted  by  our  technique  for  which  SEP  event 
onset  times  were  provided  by  Posner  (2007).  These  measures  meet  or  exceed  verification  results  for 
competing  automated  SEP  warning  techniques  but,  at  the  present  stage  of  space  weather  forecasting, 
fall  well  short  of  those  achieved  with  a  human  (aided  by  techniques  such  as  ours)  making  the  ultimate  yes/no 
SEP  event  prediction.  We  give  some  suggestions  as  to  how  our  method  could  be  improved  and  provide 
our  tlare  and  SEP  event  database  in  the  auxiliary  material  to  facilitate  quantitative  comparisons  with 
techniques  developed  in  the  future. 
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Weather,  7,  S04008,  doi:10.1029/2007SW000379. 


1.  Introduction 
1.1.  Motivation 

[2]  Solar  energetic  proton  (SEP)  events  constitute  a 
hazardous  condition  in  interplanetary  and  near-Earth 
space.  SEPs  damage  electronic  components  on  satellites 
and  produce  spurious  signals  which  can  lead  to  spacecraft 
malfunction  [Feynman  and  Gabriel,  2000;  Dyer  et  al.f  2004; 
Incci  et  al.f  2005].  SEPs  also  pose  a  radiation  threat  for 
astronauts  [Cucinotta  et  ai,  2002;  Hoff  et  ai,  2004]  and  crews 
of  high-flying  aircraft  and  commercial  airlines  in  polar 
routes  [Beck  et  ai,  2005;  Dyer  et  ai ,  2005,  Getley  et  ai,  2005]. 


'institute  of  Interplanetary  Space  Physics,  1NAF,  Rome,  Italy. 
2Spaee  Vehicles  Directorate,  AFRL.,  Hanscom  Air  Force  Base, 
Massachusetts,  USA 
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*NOAA  Space  Weather  Prediction  Center,  Boulder,  Colorado, 
USA. 
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Finally,  SEPs  can  impact  the  polar  ionosphere,  causing 
absorption  of  high-frequency  radio  waves,  thereby  affect¬ 
ing  long-distance  radio  communication  and  radar  systems 
[Hunsucker,  1992;  Hargreaves,  2005].  Hence,  a  warning 
system  is  required  in  order  to  predict  SEP  event  occur¬ 
rence  and  mitigate  their  impacts. 

1.2.  SEP  Event  Forecasting:  State  of  the  Art 
[3]  Current  methods  for  forecasting  SEPs  rely  on  obser¬ 
vations  of  associated  solar  phenomena  and  the  difference 
between  the  transit  time  of  electromagnetic  signatures  of  SEP 
acceleration /escape  and  the  time  required  for  a  >10  MeV 
proton  event  to  meet/exceed  the  NOAA  Space  Weather 
Prediction  Center  (SWPC)  SEP  threshold  of  J  (>0  MeV)  = 
10  proton  flux  units  (1  pfu  =  1  pr  cm  2  sr  1  s  J).  In  an 
alternative  approach,  Posner  [2007]  recently  developed  an 
electron-based  prediction  technique  that  exploits  the 
shorter  transit  time  of  electrons  relative  to  ions.  At  present, 
the  principal  acceleration  mechanism  for  >10  MeV  protons 
is  under  debate,  with  advocates  divided  between  CME- 
d riven  shocks  | Reatnes,  1999a;  Cliver  et  ai,  2004;  Tylka  et  ai, 
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2005]  and  a  post-CME  (flare)  reconnection  process  [e.g.. 
Cane  et  ah,  2002;  Klein  et  al,  2005;  Marque  et  ah,  2006]. 
Physics-based  numerical  models  of  SEP  acceleration  at 
shocks  [e.g.,  Roussev  et  al,  2004;  Zank  et  al,  2005;  Sokolov  et 
al,  2006]  have  not  yet  reached  the  point  where  they  can  be 
used  in  an  operational  setting.  No  research  to  date  has 
combined  accurate  simulations  of  shock  evolution,  particle 
injection  and  acceleration  and  interplanetary  transport 
[see  Laiio,  2005].  The  post-CME  reconnection  process  for 
SEP  acceleration  also  lacks  an  operational  model. 

[4]  The  present  SEP  prediction  model  used  in  opera¬ 
tions  at  NOAA  SWPC  is  called  "Protons"  \Bakh,  1999]. 
This  model,  first  implemented  in  real  time  during  the 
declining  years  of  solar  cycle  20  [Hecknum,  1979],  is  based 
on  the  standard  assumption  that  there  is  a  relationship 
between  the  intensity  of  solar  flare  emissions  and  SEP 
event  occurrence.  It  estimates  the  probability  (P)  for  the 
occurrence  of  a  SEP  event  following  a  soft  X-ray  (SXR;  1  - 
8  A)  burst  based  on  the  SXR  peak  flux  and  time-integrated 
flux,  the  occurrence  or  nonoccurrence  of  type  11  and/or 
type  IV  radio  bursts,  and  the  Hn  flare  location.  Protons 
was  recently  validated  by  Batch  [2008]  who  found  that  for 
the  period  from  1986  to  2004,  the  probability  of  detection 
(POD)  of  SEP  events  meeting  the  NOAA  SWPC  SEP  event 
threshold  was  57%  with  a  false  alarm  rate  (FAR)  of  55%. 

[5]  At  SWPC,  the  Protons  program  is  only  used  as  a 
decision  aid,  however,  and  the  final  yes/no  SEP  prediction 
is  made  by  a  forecaster.  Thus  the  SEP  event  forecasts 
issued  by  SWPC  are  significantly  more  accurate  than 
those  given  by  Protons  alone,  and,  moreover,  have  been 
improving  with  time  (http://www. swpc.noaa.gov/forecast 
verification/ProtonWarning. html).  For  the  interval  from 
1995  to  2005  on  which  the  study  presented  herein  is  based, 
the  SWPC  SEP  POD  was  88%  (78/89)  with  a  FAR  of  18% 
(17/95).  The  challenge  for  the  solar-terrestrial  community 
is  to  develop  automated/objective  computer  models  of 
equivalent  accuracy  to  those  obtained  with  forecaster 
input.  As  has  been  the  case  for  terrestrial  weather  fore¬ 
casting,  the  approach  to  this  goal  will  be  incremental 
[Siscoe,  2006].  The  present  work  is  a  step  along  that  path. 

1.3.  Approach 
1.3.1.  Guidelines 

[6]  In  developing  a  short-term  warning  system  for  SEP 
events,  we  have  followed  two  guiding  principles: 

[7]  1.  Maximize  warning  time.  Thus,  for  model  input,  we 
have  used  data  that  are  (or  can  be  made)  available  to 
SWPC  in  real  time.  Our  focus  is  on  parameters  that  are 
observable  (or  can  be  inferred)  early  in  an  event,  close  to 
flare  maximum.  While  this  does  not  preclude  using 
parameters  such  as  CME  speed,  for  example,  it  does  make 
it  imperative  that  such  speeds  be  obtained/inferred  very 
early  in  an  event.  As  we  will  show,  even  with  this  empha¬ 
sis  on  maximizing  warning  time,  the  median  lead  time  for 
the  successful  (post  hoc)  forecasts  based  on  our  technique 
is  only  55  min. 


[8]  2.  Follow  an  empirical  approach.  Basically,  we  rely 
on  the  "big  flare  syndrome"  [Kahler,  1982],  notion  that  big 
flares  have  more  of  everything.  The  larger  the  flare,  the 
more  likely  it  will  be  followed  by  a  significant  SEP  event  at 
1  AU.  The  pitfalls  of  this  flare-size-based  approach,  spe¬ 
cifically  the  occasional  occurrence  of  SWPC  level  events 
following  eruptions  with  weak  flare  emissions  (—15% 
(12/78)  of  SWPC  SEP  events  from  visible  disk  sources 
during  the  1995-2005  originated  in  eruptions  for  which 
the  associated  SXR  burst  was  <M2),  have  been  docu¬ 
mented  by  Cliver  et  al  [1983]  and  Cliver  [2006].  In  keeping 
with  our  use  of  an  empirical  approach,  we  have  used  a 
number  of  free  parameters  (as  noted  below)  arrived  at  by 
trial  and  error  to  maximize  the  forecast  results. 

[9]  Our  method  only  indicates  whether  a  solar  eruption 
will  produce  an  SWPC  SEP  event;  it  does  not  give  an 
estimate  of  the  peak  proton  intensity  or  proton  fluence. 
Such  estimates,  as  well  as  predictions  of  additional  SEP 
parameters  such  as  event  peak  time  are  provided  by  the 
Protons  model  [Batch,  1999,  2008]  and  the  Proton  Prediction 
Study  (PPS)  [ Smart  and  Shea,  1979,  1989,  1992;  cf.,  Kahler  et 
al,  2007],  albeit  with  substantial  uncertainty/inaccuracy 
\Balch,  1999]. 

1.3.2.  Inputs  to  Forecast  Model 

[10]  Our  empirical  SEP  forecast  technique  is  based 
on:  flare  location,  flare  size,  and  evidence  of  particle 
acceleration/escape. 

1.3.2.1.  Flare  Location 

[11]  It  has  long  been  recognized  that  flare  longitude  is  a 
critical  parameter  for  a  solar  eruption  to  be  followed  by  a 
SEP  event  at  Earth  [McCracken,  1962;  Van  Hollebeke  et  al, 
1975;  Cane  et  al,  1988;  Shea  and  Smart,  1990;  Belov  et  al, 
2005;  Laurenza  et  al,  2007].  Flares  located  near  the  foot 
point  of  the  spiral  magnetic  field  line  from  Sun  to  Earth 
(— W55°  heliolongitude)  are  more  likely  to  be  followed  by 
significant  SEP  events  at  1  AU  than  those  at  eastern 
longitudes. 

1.3. 2. 2.  Flare  Size 

[12]  Over  40  years  ago,  Webber  [1963]  presented  evidence 
for  a  power  law  relationship  between  time-integrated  radio 
flux  from  flares  and  the  time-integrated  SEP  flux  associ¬ 
ated  with  the  flare.  Since  then,  there  have  been  numerous 
additional  studies  linking  various  aspects  of  flare  size  to 
SEP  occurrence  [e.g.,  Castelli  et  al,  1967;  Croont,  1971a; 
Sanis  and  Shawhan,  1973;  Kahler,  1982;  Nonnast  et  al,  1982; 
Belov  et  al,  2005].  Our  forecast  scheme  uses  SXR  integrated 
flare  intensity  as  a  measure  of  flare  size  because  the  GOES 
SXR  data  are  available  at  SWPC  in  real  time.  In  our 
technique,  SEP  event  forecasts  are  only  made  for  >M2 
class  1  -8  A  flares.  While  -15%  of  the  SWPC  SEP  events  in 
our  sample  were  associated  with  <M2  SXR  flares,  the 
probability  of  a  well-connected  <M2  SXR  flare  being 
associated  with  such  a  SEP  event  is  very  small  (<10%  for 
even  Ml. 0-1. 9  events  [Belov  etui,  2005]).  Thus,  because 
our  method  is  based  on  SXR  intensity,  successful  predic¬ 
tion  of  SWPC  level  SEP  events  associated  with  <M2  class 
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flares  (increasing  the  probability  of  detection)  would  be 
accompanied  by  an  unacceptably  high  false  alarm  rate. 
1.3. 2. 3.  SEP  Acceleration  and  Escape 

[l 3 1  Low -frequency  (<1  MHz)  type  111  radio  bursts 
observed  from  space  are  timely /reliable  indicators  that 
particles  have  been  accelerated  at  the  Sun  and  have 
escaped  into  the  interplanetary  medium.  A  frequency  of 
1  MHz  corresponds  to  a  radial  distance  from  Sun  center  of 
^7  Rs  | Leblanc  et  al.,  1 9981 .  Thus  these  low-frequency 
bursts  would  seem  essential  ingredients  of  any  SEP  event 
warning  scheme.  Cane  et  al.  |2002|  reported  that  such 
events  were  the  outstanding  feature  in  low-frequency 
radio  spectra  in  essentially  all  >20  MeV  SEP  events  [see 
also  MacDowall  et  al.,  2003).  Setting  aside  the  ongoing 
debate  about  the  origin  of  the  type-ill-producing  electrons 
in  eruptions  associated  with  large  SEP  events  and  the 
proton  acceleration  mechanism  in  such  SEP  events  \Cane 
et  al.,  2002;  Cliver  et  al.,  2004;  Marque  et  al.,  2006;  Cliver  and 
Ling,  2007),  we  will  use  Wind/ WAVES  [Bougeret  et  a L,  1995) 
M  MHz  time-integrated  intensities  in  our  SEP  event  warn¬ 
ing  method  as  an  indicator  of  SEP  acceleration/escape. 
Low-frequency  radio  data  from  STEREO  are  currently 
available  in  near-real  time  at  SWPC  via  the  World  Wide 
Web  (http://wvvvv.stereo.gsfc.nasa.gov;  data  latency  of 
5  min  although  this  will  increase  as  STEREO  moves  further 
from  Earth)  with  ^70%  data  coverage  (potential  to  100%). 

1.4.  Roadmap 

|m|  In  section  2,  we  present  and  discuss  the  flare  and 
SEP  databases  for  the  solar  cycle  23  interval  used  in  this 
study.  In  section  3,  we  present  and  evaluate  our  technique 
and  in  section  4  we  summarize  and  discuss  our  results. 

2.  Database 
2.1.  SEP  List 

[  15)  We  compiled  a  list  of  SEP  events  for  solar  cycle 
23  (January  1995  to  December  2005)  from  the  5-min  proton 
data  measured  on  board  the  GOES  satellite  series  (available 
at  http://spidr.ngdc.noaa.gov/spidr/index.jsp).  We 
required  a  >10  MeV  proton  flux  MO  pfu  for  three  consec¬ 
utive  5-min  intervals  to  define  a  proton  enhancement  as  a 
SEP  event.  (When  a  SEP  event  was  observed  by  more  than 
one  GOES  spacecraft,  we  used  the  data  from  the  satellite 
which  recorded  the  largest  event  peak  intensity.  Eor  the 
large  majority  of  events,  intensity  differences  between 
spacecraft  are  small.)  The  threshold  of  10  pfu  is  approxi¬ 
mately  2  orders  of  magnitude  above  the  normal  back¬ 
ground  and  represents  the  lowest  intensity  where 
radiation  hazard  analysis  is  needed.  We  did  not  consider 
solar  particle  enhancements  which  did  not  meet  this 
threshold  in  our  analysis,  e.g.,  a  forecast  of  a  SEP  event 
by  our  technique  followed  by  a  >10  MeV  event  with  peak 
intensity  of  9  pfu  was  considered  to  be  a  false  alarm.  For 
times  when  the  >10  MeV  proton  background  was  >10  pfu, 
we  required  an  increase  by  a  factor  of  two  to  register  a  new 
SEP  event. 


|ib)  We  associated  each  identified  SEP  event  with  a 
solar  source,  either  (1)  a  visible  disk  flare  or  disappearing 
filament  or  (2)  a  partially  observed,  or  inferred,  behind- 
the-limb  eruption.  In  making  flare  associations,  we  aimed 
to  separate  the  proton  enhancements  from  different  solar 
flares  to  the  best  of  our  ability.  We  attributed  each  SEP 
increase  above  the  SWPC  threshold  (or  each  increase  of  a 
factor  of  two  above  a  >10  pfu  background)  to  a  single  solar 
event.  For  example  if  two  flares,  A  and  B,  were  followed  by 
an  increase  to  7  pfu  and  a  second  increase  (from  7  pfu)  to 
12  pfu,  we  would  assign  the  SWPC  SEP  event  to  flare 
B.  This  is  a  necessary  simplification  for  a  prediction 
scheme  because  a  yes/no  SEP  event  occurrence  decision 
must  be  made  on  every  SXR  flare  which  exceeds  the 
threshold  (M2  in  our  case)  for  making  a  forecast. 

[17)  From  1995  to  2005,  we  identified  93  prompt  SEP 
events  meeting  the  NOAA  criteria  for  a  significant  event. 
Our  definition  of  "prompt"  includes  SEP  events  for 
which  the  onset  (e.g.,  15  July  2002)  and/or  the  peak 
(e.g.,  17  November  2001)  may  have  been  delayed  from  the 
flare  because  of  propagation  effects  (e.g.,  15  March  2002)  as 
long  as  it  appeared  that  the  rise  exceeded  the  >10  pfu 
threshold  outside  the  time  of  any  shock  spike.  We  did 
not  include  six  SEP  events  which  only  met  the  SWPC 
threshold  as  a  result  of  shock  spikes  (associated  with  solar 
eruptions  on  23  September  1998,  20  January  1999,  3  May 
1999,  9  August  2000,  26  April  2001,  and  12  September 
2004).  Similarly  we  did  not  consider  shock  spikes,  as  well 
as  modulations  caused  by  nonshock  solar  wind  structures, 
associated  with  a  prompt  event  on  our  list  (e.g.,  solar  events 
on  26  December  2001,  18  March  2002,  and  7  November 
2004).  Advance  warning  for  such  shock  spikes  or  energetic 
storm  particle  (the  so-called  ESP)  events,  which  can  have 
very  intense  >10  MeV  peaks  | Reames,  1999b),  may  be  pro¬ 
vided  by  shock/SEP  monitoring  from  LI  [Cohen  et  al.,  2001). 

| is)  Attribution  of  SEP  events  to  solar  sources  was  a  critical 
part  of  our  study.  For  most  of  these  large  (J  (>10  MeV)  > 
10  pfu)  SEP  events,  the  process  is  straightforward  but  diffi¬ 
culties  do  arise  for  behind-the-limb  sources  as  well  as  for 
events  for  which  the  Sun  was  poorly  observed.  Generally, 
we  identified  the  largest  soft  X-ray  flare  near  the  SEP  event 
start  time  as  the  SEP  source,  taking  into  consideration  time 
coincidence  with  type  11  and  IV  events  and  fast  CMEs  (all  in 
accordance  with  the  big  flare  syndrome,  as  well  as  the 
current  CM E-d riven  shock  picture  of  large  SEP  events). 
Despite  our  use  of  the  various  mentioned  observables  (e.g., 
type  Us,  IVs,  and  CMEs)  in  associating  the  SEP  events  in 
Table  1  with  solar  eruptions,  our  warning  technique  relies 
only  on  the  parameters  (flare  location,  size,  and  evidence 
for  particle  acceleration/escape)  specified  in  section  1.3.2. 
Active  region  histories  were  an  important  factor  in  attrib¬ 
uting  SEP  events  to  solar  backside  sources;  coronagraph, 
extreme  ultraviolet  (EUV)  and  soft  X-ray  images  in  the 
LASCO  CME  catalog  (http://cdaw.gsfc.nasa.gov/CME  list/) 
were  also  particularly  helpful  for  identifying  backside 
sources.  To  obtain  our  final  list  of  SEP  events,  we  made 
two  independent  determinations  of  SEP  events  and  their 
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Table  1.  SEP  Flare  List3 


Event 

Number 

SXR  Date 

Peak 

Time 

SXR 

Class 

Ha 

Location 

SXR  FI uence 
Used  0/m2) 

SXR 

Flas 

Wind/Waves 
Fluencc  (sfu  x  min) 

Frequency 

(kHz) 

Delay  to  Onsetb 
(min) 

SEP  Forecast 
Result 

1 

20  Oct.  1995 

0606 

Ml. 7 

S11VV53 

3.28E-2 

5 

5.99E+5 

940 

MISS 

2 

4  Nov.  1997 

0558 

X2.1 

S15VV34 

5.86E  2 

7 

1.20E+7 

940 

36 

Hit 

3 

6  Nov.  1997 

1155 

X9.4 

S18YV63 

3.61E-1 

7 

1  87E+7 

940 

61 

Hit 

4 

20  Apr.  1998 

1021 

Ml  .5 

W115 

5 

2  May  1998 

1342 

XI. 2 

S15VV15 

7.37E-2 

5 

2.14E+7 

940 

56 

Hit 

6 

6  May  1998 

0809 

X2.8 

S11VV65 

2.35E  1 

5 

8.85E+6 

940 

13 

Hit 

7 

9  May  1998 

0340 

M7.7 

YV100 

1.08E  1 

5 

2.69E+6 

940 

Hit 

8 

24  Aug.  1998 

2212 

XI  1 

N35E09 

1.88E  1 

5 

1.79E+7 

940 

Hit 

9 

30  Sep.  1998 

1348 

M3.0 

N23VV81 

9.61E-2 

2 

7.09  E +5 

940 

Hit 

10 

14  Nov.  1998 

0518 

Cl  .8 

VV130 

11 

24  Apr.  1999 

1300 

<C1 

YV130 

12 

1  Jun  1999 

1845 

C? 

>\V90 

13 

4  Jun.  1999 

0703 

M4.2 

N17VV69 

2.62E-2 

5 

3.95E+6 

940 

Miss 

14 

18  Feb.  2000 

0925 

C? 

W120 

15 

4  Apr.  2000 

1539 

Ml.O 

N15YV63 

3.30E-2 

2 

9.24E+6 

940 

MISS 

16 

6 Jun  2000 

1525 

X2.5 

N21E15 

4.22E-1 

5 

1.28E+7 

940 

205 

Hit 

17 

10  Jun.  2000 

1700 

M5.6 

N22W39 

1.02E  1 

5 

9.57E+6 

940 

20 

Hit 

18 

14  JuL  2000 

1023 

X6.1 

N22VV07 

1.35E+0 

5 

1  20E+7 

940 

15 

Hit 

19 

22  Jul  2000 

1132 

M3. 9 

N14VV56 

8.18E-2 

5 

1.69E+5 

940 

Miss 

20 

27  Jul.  2000 

1930 

C? 

>VV  1 20 

21 

12  Sop.  2000 

1212 

Ml.O 

S19YV08 

2.94  E  2 

1 

5.43E+6 

940 

MISS 

22 

16  Oct.  2000 

0735 

M2. 8 

VV110 

8.54E-  2 

1 

7.18E+4 

940 

Miss 

23 

25  Oct.  2000 

1109 

C4  1 

>VV90 

24 

8  Nov.  2000 

2327 

M7.9 

N10VV77 

3.36E-1 

3 

4.51  E+6 

940 

-3 

Hit 

25 

24  Nov.  2000 

1513 

X2.5 

N21VV08 

1.64E-1 

5 

6.77E+6 

940 

Hit 

26 

25  Nov.  2000 

0131 

M8.4 

N07E50 

2.66E-1 

5 

1.69E+6 

940 

Miss 

27 

28  Jan.  2001 

1558 

Ml. 7 

S04VV59 

3.54E  2 

5 

1.60E+6 

940 

MISS 

28 

29  Mar.  2001 

1015 

XI.  8 

N14YV13 

2.74E-1 

5 

3.83E+5 

940 

Miss 

29 

2  Apr.  2001 

2150 

X18.4S 

N18W82 

1.62E+0 

5 

2.75E+6 

940 

Hit 

30 

10  Apr.  2001 

052b 

X2.3 

S23YV09 

3.66E-1 

5 

9.50E+6 

940 

Hit 

31 

12  Apr.  2001 

1028 

X2.2 

S19YV43 

4.02E  1 

5 

6.54E+6 

940 

Hit 

32 

15  Apr.  2001 

1350 

X15.8 

S20W85 

6.20E  1 

7 

8.77  E+6 

940 

28 

Hit 

33 

18  Apr.  2001 

0214 

C2.4 

YV120 

34 

7  May  2001 

1220 

C4.1 

VV35 

1.22E-2 

5 

1  50E+4 

940 

MISS 

35 

15  Jun.  2001 

1530 

<C2 

W1 15 

36 

9  Aug.  2001 

1122 

C3.9 

W10 

LOSE  2 

5 

1.57E+4 

940 

MISS 

37 

15  Aug.  2001 

2355 

<Cl 

Wl  80 

38 

15  Sep.  2001 

1128 

Ml  .6 

S21 YV49 

5.35  E -2 

2 

1.98E+4 

940 

MISS 

39 

24  Sep.  2001 

1035 

X2.7 

S17E29 

1.09E+0 

3 

1.48E+6 

940 

75 

Hit 

40 

1  Oct.  2001 

0515 

M9.1 

S22YV85 

7.56E-2 

5 

1.12E+5 

940 

Miss 

41 

19  Oct.  2001 

1630 

XI. 8 

N15YV29 

1.66E-1 

5 

3.38E+4 

940 

Miss 

42 

22  Oct.  2001 

1508 

M7.0 

S17E19 

1.89E-1 

5 

1.77E+7 

940 

24 

Hit 

43 

4  Nov.  2001 

1619 

XI. 1 

N07W19 

2.76E-1 

2 

1  36E+7 

940 

39 

Hit 

44 

17  Nov.  2001 

0523 

M3.0 

S13E42 

1  34E-1 

3 

3.69E+6 

940 

Miss 

45 

22  Nov.  2001 

2034 

M4.1 

S25VV67 

6.49E  2 

5 

4.74E+6 

940 

Hit 

46 

22  Nov.  2001 

2327 

X1.0 

S15YV34 

4.68E  1 

3 

1.38E+5 

940 

Hit 

47 

26  Dec.  2001 

0536 

M7.6 

N08VV54 

6.30E  1 

4 

1.14E+6 

940 

6 

Hit 

48 

28  Dec.  2001 

2042 

X3.5 

S26E95 

2.92E+0 

4 

4.43E+6 

940 

Hit 

49 

8  Jan.  2002 

2025 

C9.6 

N05YVr95 

50 

14  Jan  2002 

0623 

M4.8 

YV90 

4.03E  1 

4 

9.69  E +4 

940 

115 

Hit 

51 

20  Feb.  2002 

0612 

M5.7 

N12YV72 

1.75E  2 

7 

7.40E+6 

940 

Miss 

52 

15  Mar.  2002 

2306 

M2.3 

SOS  W  03 

6.34E-2 

1 

2.15E+6 

940 

Miss 

53 

18  Mar.  2002 

0230 

Ml.l 

VV22 

1.73E— 2 

5 

2.67E+5 

940 

MISS 

54 

22  Mar.  2002 

mi 

Ml. 8 

S10VV95 

55 

17  Apr.  2002 

0824 

M2. 9 

S14YV36 

1.35E-1 

3 

6.93E+5 

1000 

86 

Hit 

5b 

21  Apr.  2002 

0147 

XI. 7 

S14YV84 

7.82E-  1 

3 

4.51E+6 

940 

-7 

Hit 

57 

22  May  2002 

0348 

C5.2 

S22YV53 

1.82E  2 

lh 

2.02E+6 

940 

MISS 

58 

7  Jul.  2002 

1143 

Ml  2 

Wl  15 

59 

15  Jul.  2002 

2008 

X3.2 

N19YV01 

1.49E  1 

7 

9.81  E+6 

940 

882 

Hit 

60 

20  Jul.  2002 

2128 

X3.4 

F.100 

1.08E+0 

5 

3.32E+6 

940 

Hit 

61 

14  Aug.  2002 

0211 

M2. 6 

N10YV54 

1.06E  1 

3 

9.51  E+5 

940 

9 

Hit 

62 

22  Aug.  2002 

0157 

M5.9 

S07VV62 

3.82E  2 

5 

1.02E+7 

940 

Miss 

63 

24  Aug.  2002 

0111 

X3.5 

S02YV81 

5.75  E  1 

5 

Cal 

940 

64 

5  Sep.  2002 

1704 

C5.2 

N09E28 

2.49E  2 

3 

2.34E+5 

940 

MISS 

65 

9  Nov.  2002 

1323 

M4.9 

S12YV29 

5.52E  2 

5 

8.14E+6 

940 

55 

Hit 

66 

28  May  2003 

0027 

X3.9 

S07W21 

3.12E  1 

5 

7. 20  E+6 

940 

Hit 
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Table  1.  (continued) 


Event 
Nu  mbcr 

SXR  Date 

Peak 

Time 

SXR 

Class 

Ha 

location 

SXR  Pluence 
Used  (J/ni2) 

SXR 

Hag 

Wind/ Waves 
Fluence  (sfu  x  min) 

Frequency 

(kH/) 

Delay  to  Onset1’ 
(min) 

SEP  Forecast 
Result 

e>7 

31  May  2003 

0224 

XI  .0 

S07VV65 

1.20E-1 

5 

7.96E+6 

940 

Hit 

68 

17  Jun.  2003 

2245 

M6.8 

S08E61 

Gap 

7 

8.56E+4 

940 

69 

26  Oct.  2003 

1811 

XI. 4 

N02VV38 

3.83E-1 

1 

1.43E+6 

916 

Hit 

70 

28  Oct.  2003 

1110 

X18.4S 

S16E07 

1.96E+0 

5 

2.16E+7 

916 

Hit 

71 

29  Oct.  2003 

2049 

X10.8 

S15VV02 

9.80  E-  1 

5 

8.79E+6 

916 

Hit 

72 

2  Nov.  2003 

1725 

X9.3 

S14VV56 

1 .09E+0 

5 

2.70E+6 

916 

Hit 

73 

4  Nov.  2003 

1944 

X18.4S 

S19VV83 

2.65E+0 

1 

9.53E+5 

916 

Hit 

74 

20  Nov.  2003 

2353 

M6.2 

N02W17 

2.82E  2 

7 

7.07  F. +6 

916 

Miss 

75 

2  Dec.  2003 

1150 

C8.5 

S17W95 

76 

11  Apr.  2004 

0419 

Ml  .0 

S14VV47 

1.72E  2 

5 

3.03E+6 

940 

MISS 

/  / 

25  Jul.  2004 

1515 

Ml. 2 

N08VV33 

3.25 E  -2 

1 

7.51  E+4 

940 

MISS 

78 

19  Sep.  2004 

1711 

M2.0 

N05VV58 

5.46E  2 

5 

4.20E+6 

940 

Hit 

79 

1  Nov.  2004 

0555 

<C1 

>YV90 

80 

7  Nov.  2004 

1606 

X2.2 

N09VV17 

2.08E  1 

5 

1.36E+6 

940 

Hit 

81 

10  Nov.  2004 

0213 

X2.8 

N09VV49 

1 .68  E  1 

7 

1.84E+6 

940 

Hit 

82 

15  Jan  2005 

2300 

X2.9 

N14VV08 

8.63E  1 

2 

1.01E+6 

916 

Hit 

S3 

17  Jan.  2005 

0952 

X4.2 

N14VV24 

7.20E  1 

5 

1.63E+6 

916 

Hit 

84 

20  Jan.  2005 

0700 

X7.9 

N12YV58 

1.97E+0 

5 

1  66E+7 

916 

Hit 

85 

13  May  2005 

1657 

M8.5 

N12E11 

2.50E-1 

5 

1.79E+7 

916 

Hit 

86 

16  Jun.  2005 

2022 

M4.3 

N09YV87 

7.75E  2 

5 

6.94  E +5 

916 

Miss 

87 

13  Jul.  2005 

1449 

M5.6 

N13YV75 

4.64E  1 

4 

1  08E+5 

916 

Hit 

88 

14  Jul  2005 

1054 

XI  .4 

YV95 

6.63  E  1 

3 

2.65E+4 

916 

Miss 

89 

17  Jul.  2005 

1120 

<1 

YV135 

90 

27 Jul  2005 

0501 

M3. 8 

<E90 

1.16E  1 

5 

8.83  E+4 

916 

Miss 

91 

22  A  ur.  2005 

1728 

M6.2 

S12YV60 

2.87E  1 

3 

1 54E+6 

916 

Hit 

92 

7  Sep.  2005 

1740 

X18.1 

S06E89 

6.65E+0 

3 

1.42E+7 

916 

Hit 

93 

13  Sep.  2005 

2004 

XI. 6 

S09E05 

4.86E  1 

5 

1 49E+5 

916 

Miss 

“Hero  J  (>10  MeV)  >  10  pfu. 

L\  allies  based  on  published  SEP  onset  times  are  by  Posner  120071  The  listed  time  does  not  include  the  time  required  for  the  event  to  rise  above 
the  SWPC  10  pfu  prediction  threshold  nor  the  15  min  required  to  certify  a  new  SEP  event 


solar  associations  (one  by  Laurenza,  Cliver,  and  Hewitt  and 
one  by  Balch)  and  reconciled  discrepancies,  also  taking  into 
account  a  third  list  of  SEP  events  (based  on  the  NOAA  list  at 
http://umbra.nascom.nasa.gov/SEP/)  with  associations  in¬ 
dependently  determined  by  Cane  et  al.  [20061. 

[  19)  Our  final  reconciled  list  of  SEP  events  with  their 
solar  associations  is  reported  in  Table  1.  The  information 
in  the  1  able  1  is  as  follows:  (1 )  event  number;  (2)  event  date 
(of  the  associated  solar  event);  (3)  peak  time  of  soft  X-ray 
burst  (when  the  SXR  class  is  indefinite,  this  time  refers  to 
the  onset  of  the  1MHz  emission;  no  SXR  integration  was 
attempted  for  these  events);  (4)  soft  X-ray  burst  class 
(saturated  events  flagged  with  an  "S"  in  which  case 
(3)  refers  to  the  onset  of  saturation);  (5)  heliographic 
location  of  associated  solar  eruption  (from  National  Geo¬ 
physical  Data  Center  [2006]  or  determined  from  SXR  and/ 
or  EUV  images  in  the  LASCO  CME  catalog);  (6)  time- 
integrated  SXR  intensity  (nominally  (see  section  3.1.2) 
taken  between  the  1/3  power  points  on  the  rise  and  decay 
of  the  event;  no  SXR  integration  was  made  for  <M2.0 
events  from  beyond  the  limb);  (7)  SXR  integration  flag 
(defined  in  section  3.1.2);  (8)  time-integrated  1  Ml  lz  Wind/ 
WAVES  type  111  intensity  from  10  min  before  the  onset  of 
SXR  event  to  10  min  after  the  SXR  burst  peak;  (9)  Waves 
frequency  used;  (10)  delay  time  from  SXR  peak  +10  min 
(time  of  forecast)  to  the  >30  MeV  SEP  event  onset  (taken 
from  Posner  |2007| );  and  (11)  SEP  Forecast  Result  ("Hit", 
SEP  event  correctly  predicted;  "Miss",  SEP  event  from 


frontside  or  backside  source  with  peak  intensity'  >M2  not 
predicted;  no  entry  indicates  either  that  the  responsible 
eruption  was  located  on  the  backside  of  the  Sun  and  the 
associated  SXR  flare  w'as  <M2  (16  cases)  or  that  SXR  or 
radio  data  w'ere  unavailable  (2  cases)).  Note  that  while  no 
prediction  was  made  for  SEP  events  associated  with  vis¬ 
ible  disk  eruptions  for  which  the  associated  SXR  flare  was 
<M2,  such  events  were  assigned  to  the  Miss  class  (given  as 
"MISS"  in  Table  1  to  distinguish  them  from  the  normal 
Miss  events  associated  with  >M2  flares)  because,  unlike 
the  small  (<M2)  behind-the-limb  flares,  the  model  inputs 
in  these  cases  were  not  compromised.  SEP  events  associ¬ 
ated  with  small  (<M2)  behind-the-limb  (>90°  longitude) 
flares  are  not  counted  as  misses  because  the  required 
input  data,  as  the  accurate  measurement  of  the  integrated 
SXR  intensity,  is  not  available  because  of  likely  limb 
occultation.  Thus  it  seems  fair  to  not  count  these  events 
as  misses  in  much  the  same  sense  that  we  exclude  events 
with  data  gaps.  At  the  same  time,  we  issued  a  hypothetical 
forecast  for  every  M2  SXR  flare,  regardless  of  flare 
location.  It  may  be  possible  (and  is  certainly  desirable)  to 
improve  forecasts  of  SEP  events  from  behind  the  limb 
sources  by,  e.g.,  using  STEREO  type  observations  that  look 
beyond  the  limb  or  with  a  particle-based  technique  such 
as  that  of  Posner  [2007],  but  such  an  extension  is  beyond 
the  scope  of  the  present  paper. 

|2o|  Figure  1  gives  the  longitudinal  distribution  of  the 
93  prompt  SEP  events  (using  the  best  guess  longitude  for 
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SEP  Flares 


Figure  1.  Flare  heliolongitudes  (in  15°)  bins  for  the  93 
SVVPC  prompt  SEP  events  during  cycle  23  (1995-2005). 

SEP  events  originating  on  the  backside  of  the  Sun).  The 
familiar  western  bias  of  such  events  is  apparent. 

2.2.  Soft  X-Ray  List 

[21 1  From  1995  to  2005,  we  identified  704  >M2  SXR 
flares  in  the  GOES  1-min  averaged  1-8  A  data  (available 
at  http://spidr.ngdc.noaa.gov/spidr/index.jsp).  In  making 
the  flare  associations  for  the  >M2  soft  X-ray  events,  we 
used  the  list  of  Ho  flares  from  National  Geophysical  Data 
Center  [2006]  in  conjunction  with  the  daily  soft  X-ray  plots. 
For  events  with  no  reported  Ha  flare,  we  made  associa¬ 
tions  using  the  EIT  and  SXT  data  from  the  SOHO  LASCO 
CME  catalog  as  well  as  active  region  histories  and  the 
narrative  texts  in  The  Weekly:  Preliminary  Report  and 
Forecast  of  Solar  Geophysical  Data  (Space  Weather 
Prediction  Center,  available  at  http://www.sec.noaa.gov/ 
weekly/index.html;  for  these  events  only  longitudes  are 
listed  in  the  flare  location).  The  Boulder  reports  were 
particularly  helpful  in  the  sense  that  they  often  indicated 
sources  of  SXR  flares  that  lacked  an  Ha  counterpart,  giving 
the  forecaster's  best  guess  based  on  the  data  available  at  the 
time  and  thus  injecting  realism  into  the  exercise,  since  these 
educated  guesses  would  be  folded  into  our  technique 
evaluation.  As  was  done  for  the  SEP  events  in  Table  1,  the 
list  of  X-ray  events  and  associations  was  compiled  sepa¬ 
rately  by  two  subgroups  of  coauthors  (Laurenza,  Cliver,  and 
Hewitt  and  Balch  (1995-2004))  and  then  reconciled,  Be¬ 
cause  we  used  the  largest  peak  SXR  intensity  observed  by 
any  GOES  satellite,  our  list  includes  55  more  events  (all  with 
intensity  classes  <M2.6;  51  <M2.2)  than  that  compiled  by 
Balch  [2008]  for  the  1995-2004  period  of  overlap. 

|22)  The  list  of  704  SXR  flares  is  given  in  Data  Set  SI  in 
the  auxiliary  material.1  The  information  in  Data  Set  SI  is  as 

'Auxiliary  material  data  sets  are  available  at  ftp://ftp.atfu.org/ 
apend/sw 72007SVV000379.  Other  auxiliary  materials  files  are  in  the 
HTML. 


follows:  (1)  event  number;  (2)  date  of  the  SXR  flare;  (3)  peak 
time  of  SXR  flare;  (4)  peak  SXR  classification  (saturated 
events  flagged  with  an  S  in  which  case  (3)  refers  to  the 
onset  of  saturation);  (5)  flare  location;  (6)  time-integrated 
SXR  intensity  (nominally  between  the  1/3  power  points  on 
rise  and  fall  (see  section  3.1.2));  (7)  ratio  of  the  SXR 
intensity  at  the  peak  time  +10  min  (II0)  to  the  SXR  intensity 
at  the  event  peak  (IP)  (see  section  3.1.2);  (8)  extrapolated/ 
adjusted  time-integrated  SXR  intensity  (see  section  3.1.2); 
(9)  SXR  integration  flag  (defined  in  section  3.1.2);  (10)  ratio 
of  (8)  to  (6);  (11)  time-integrated  SXR  intensity  used  in 
analysis  (taken  from  either  (6)  or  (8));  (12)  time-integrated 
1  MHz  Wind/ WAVES  intensity  from  10  min  before  the 
onset  of  integration  for  the  SXR  burst  to  10  min  after  the 
SXR  burst  peak;  (13)  waves  frequency  used;  and  (14)  SEP 
forecast  result  (Hit,  SEP  event  correctly  predicted;  Miss, 
SEP  event  from  frontside  or  backside  source  with  peak 
intensity  >M2  not  predicted;  "False  Alarm",  SEP  event 
predicted  but  none  occurred;  "Correct  Null",  i.e.,  no  SEP 
event  predicted  and  none  occurred);  a  blank  indicates  a 
gap  either  in  the  SXR  or  radio  data  or  that  the  radio  burst 
occurred  during  calibration. 

[23]  Figure  2  gives  the  longitude  distribution,  binned  by 
15°,  of  all  the  >M2  flares  in  Data  Set  SI.  Note  that,  in 
comparison  with  the  longitudinal  distribution  of  all  Ha 
flares  [ Wright ,  1980],  the  distribution  in  Figure  2  is  rela¬ 
tively  flat  across  the  solar  disk.  This  reflects  both  the  large 
size  of  >M2  SXR  bursts,  facilitating  observation  of  their 
Ha-  counterparts  for  foreshortened  near-limb  events,  and 
the  use  of  imaging  data  from  Yohkoh  SXT  and  SOHO  EIT 
to  augment  the  Ho  data. 

3.  Analysis 

3.1.  Input  Parameters 

[24]  As  noted  in  section  1.3.2,  we  used  the  following 
three  parameters  as  input  in  our  forecast  scheme:  (1)  flare 


Figure  2.  Flare  heliolongitudes  (in  15°)  bins  for  the  704 
>M2  SXR  flares  events  during  cycle  23  (1995  —  2005). 
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longitude;  (2)  time-integrated  SXR  intensity;  and  (3)  time- 
integrated  MHZ  radio  intensity. 

3.1.1.  Flare  Longitude 

[ 25 ]  Comparison  of  Figures  1  and  2  reveals  the  much 
Iov\cr  probability  of  SWPC  SEP  events  arising  in  eastern 
versus  western  hemisphere  activity.  For  the  1995  —  2005 
interval,  15  SEP  events  and  327  >  M2  flares  originated  in  the 
eastern  solar  hemisphere  giving  a  5%  (15/327)  SEP  event 
"vield";  in  the  western  hemisphere  the  corresponding 
percentage  is  21%  (78/377).  Hence,  in  our  forecast  tech¬ 
nique,  we  consider  three  heliolongitude  ranges  (El20°- 
E41  ,  E40°-VV19°;  W20°-W120°)  separately,  with  the 
number  and  ranges  of  these  longitude  bins  determined 
by  trial  and  error  in  order  to  optimize  the  prediction 
results. 

3.1.2.  Time-Integrated  Soft  X-Ray  Intensity 

1 |  It  is  well  established  | Croom,  1971b;  Nonnast  et  ill., 
1982;  Koclurrov  et  a!.,  1983;  Cane  et  ai,  1986]  that  large  SEP 
events  are  preferentially  (although  not  exclusively  [e.g., 
Kahler  et  al.,  19911)  associated  with  flares  with  long  time 
scales.  Because  of  the  need  to  maximize  warning  time  for 
a  SEP  event,  however,  it  is  not  practicable  to  integrate 
over  the  full  duration  of  such  flares,  which  can  last  for 
hours.  To  obtain  a  representative  measure  of  the  full 
integrated  intensity,  we  decided  to  integrate  (after  a  trial 
and  error  process)  between  the  1/3  power  points  on  the 
rise  and  fall  of  the  SXR  burst.  In  33%  (229)  of  the  704 
cases  in  Data  Set  Si,  the  SXR  intensity  fell  to  1/3  of  its 
maximum  value  within  10  min  of  the  time  of  flare 
maximum  (these  events  are  indicated  by  a  blank  in 
extrapolated  SXR  entries  and  a  "7"  in  SXR  flag  entries 
of  Data  Set  SI).  For  the  remaining  67%  of  the  bursts, 
the  time  to  drop  to  this  level  exceeded  10  min.  For 
these  cases  (indicated  by  values  of  1-6  in  SXR  flag 
entries  of  Data  Set  SI),  wo  fit  an  exponential  curve  over 
tlie  five  1-min  data  points  spanning  minutes  6  through 
10  following  the  burst  maximum  and  extrapolated  this 
curve  to  the  1/3  power  point  to  obtain  the  SXR  integrated 
intensity  used  in  our  analysis.  If  a  complete  or  significant 
data  gap  occurred  during  the  time  of  interest  in  a  given 
event,  the  SXR  fluence  is  given  as  "Gap"  in  Table  1  and 
Data  Set  SI. 

1 2" ]  Our  SXR  tlux  integration  technique  for  events  that 
did  not  fall  to  the  1/3  power  point  within  10  min  requires 
two  footnotes.  First,  saturation  can  produce  flat  traces  for 
the  interval  from  6  to  10  min  after  burst  peak  while 
blended  peaks  can  result  in  rising  profile  (yielding  I10/Ip 
ratio  +1  and  >1,  respectively).  For  these  special  cases  (15  in 
all)  we  assigned,  after  a  process  of  trial  and  error,  a  decay 
slope  (coefficient  in  the  exponent)  that  produced  a  25% 
drop  from  the  measured  SXR  peak  intensity  after  10  min. 
Second,  inspection  showed  that  events  with  a  high  (>0.85) 
ratio  of  the  (fitted)  SXR  intensity  at  the  peak  time  +10  min 
(11(1)  to  the  (fitted)  SXR  intensity  at  the  time  of  the  event 
peak  (I|>)  (this  ratio  is  given  by  the  extrapolated  SXR  of 
Data  Set  SI)  tended  to  have  time-integrated  intensities 


that  were  overestimates.  To  adjust  for  this  effect,  we 
applied  the  following  prescription  (again  arrived  at  by 
trial  and  error):  (1)  if  0.85  <  ho/Ip  <  0.90,  then  I io/Ip  = 
0.85;  (2)  if  0.90  <  I10/IP  <  0.95,  then  I,0/IP  =  0.90;  and  (3)  if 
0.95  <  I’jq/ Ip  <  1.0,  then  I io/Vp  =  0.95.  No  adjustments  were 
made  for  I10/Ip  <  0.85.  When  calculating  the  SXR  fluence 
of  events  for  which  extrapolation  was  required,  we  began 
the  (forward)  integration  in  all  cases  from  the  peak  time  of 
the  event.  If  no  adjustment  was  required  (i.e.,  I,o/Ip  < 
0.85),  we  integrated  under  the  actual  data  points  until  the 
peak  time  +10  min  and  under  the  fitted  line  thereafter.  For 
events  requiring  adjustment,  we  integrated  under  an 
extrapolated  curve  with  prescribed  slope  starting  from 
the  measured  peak  time  and  intensity  of  the  event  and 
ending  when  the  fitted  intensity  dropped  below  1/3  of  the 
peak  intensity. 

1  2hJ  For  all  the  SXR  events  in  Data  Set  SI,  we  made  a 
determination  of  the  actual  integrated  flux  between  the  1/3 
power  points,  in  order  to  assess  the  reliability  of  our 
extrapolation/adjustment  technique.  This  integration  can 
be  problematic  for  (1)  small  (i.e.,  ~-M2)  bursts  on  an 
elevated  background,  (2)  "blended"  SXR  flares  which 
appear  as  long  decay  events,  and  (3)  a  few  genuine  slow 
rise  and  fall  events.  For  such  events,  with  unusually  long 
(artificial  or  natural)  rise  or  decay  times,  we  arbitrarily 
began  the  integration  60  min  before  the  burst  peak 
(21  cases)  and/or  ended  it  within  300  min  after  the  burst 
peak  (1  case).  The  effect  of  the  extrapolations  and  adjust¬ 
ments  can  be  seen  in  Figure  3,  which  contains  a  seatterplot 
of  the  ratio  of  the  SXR  fluence  used  in  the  analysis  to  the 
measured  time-integrated  SXR  intensity  between  the  1/3 
power  points  (used  fluence/measured  fluence  in  Data  Set 
SI)  versus  Iio/Ip  (extrapolated  SXR  in  Data  Set  Si)  before 
(top)  and  after  (bottom)  the  adjustments  for  positive 
slopes  and  high  Il0/Ip  ratios  were  applied.  It  can  be  seen 
that  the  various  adjustments  applied  move  the  ratio  of 
used  to  measured  fluence  closer  to  one  for  high  liu/Ip 
ratios,  as  desired.  In  all,  33  events  lie  more  than  a  factor 
of  two  away  from  a  ratio  of  one,  i.e.,  with  values  <0.5  or 
>2.0.  In  general,  events  with  low  values  of  the  extrapo¬ 
lated  (adjusted)/measurcd  SXR  fluence  (range  from  0.03 
to  0.48;  17  cases)  are  not  a  problem,  because  they  indicate 
that  the  extrapolation  has  effectively  removed  the  contri¬ 
bution  from  a  subsequent  closely  spaced  peak  which 
would  be  evaluated  separately.  Events  with  large  values 
of  this  ratio  (range  from  2.01  to  5.19;  16  cases)  can  result 
when  a  flat-topped  burst  drops  sharply  after  the  peak 
+10  min  forecast  time.  The  effect  of  both  underestimates 
and  overestimates  of  the  SXR  fluence  (resulting  from  our 
extrapolation/adjustment  technique)  will  be  considered 
in  section  3.4.1. 

[2m|  To  note  the  fact  that  the  integrated  SXR  intensities 
tor  the  cases  described  above  involving  extrapolation  as 
W'ell  as  ad  hoc  adjustments,  vve  included  a  SXR  integration 
flag  in  Table  1  and  Data  Set  Si.  The  values  of  this  flag, 
the  circumstances  in  which  it  applies,  the  corresponding 
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Figure  3.  Scatterplot  for  all  events  in  Data  Set  SI  of  the  ratio  of  the  SXR  fluence  used  in  our 
analysis  to  the  measured  time-integrated  SXR  intensity  between  the  1/3  power  points  (used 
fluence/measured  fluence  in  Data  Set  SI)  versus  I^/lp  (extrapolated  SXR  in  Data  Set  SI)  (top) 
before  and  (bottom)  after  the  adjustments  for  positive  slopes  and  high  I^/Ip  ratios  were  applied. 
Only  points  with  SXR  flags  in  Data  Set  SI  (SXR  flag)  with  values  from  1  to  6  are  plotted.  Red  points 
indicate  SEP  events  for  which  a  forecast  would  have  been  made. 


adjustment,  and  the  number  of  cases  of  each  type  of  event 
are  given  in  Table  2. 

3.1.3.  Time-Integrated  1  MHz  Radio  Intensity 

[3()|  The  Wind/WAVES  data  were  taken  from  the  web¬ 
site  http://lep694.gsfc.nasa.gov/waves/waves.html.  The 
data  obtained  from  the  Wind/Waves  website  are  provided 
in  terms  of  ratio  (R)  of  the  radio  flux  to  background  with 
the  background  (B)  provided  in  units  of  /iV/Hz1  2.  The 
radio  flux  in  solar  flux  units  (1  sfu  =  solar  flux  unit  = 
10“22  VV  m  2  Hz  l)  is  then  J(sfu)  =  1010(R*B)2/(ZoA),  where 
Z()  is  the  characteristic  impedance  of  free  space  (Z0  = 
377W),  and  A  is  the  area  of  the  RADI  antenna  (1225  m2). 
For  much  of  the  1995  —  2005  interval  considered,  940  kHz 
was  the  closest  frequency  to  1  MHz  at  which  measure¬ 
ments  were  routinely  made.  For  a  ~-3  month  interval  at  the 
end  of  2003,  however,  and  for  all  of  2005,  the  nearest 
frequency  was  916  kHz.  For  12  other  events  that  were 
randomly  distributed  in  time  the  closest  usable  frequency 
was  either  1000  or  1012  kHz.  For  1-min  averages  of  data  at 
these  various  frequencies  near  —1  MHz,  spectral  differ¬ 
ences  should  be  small. 

|3i|  Because  the  1  MHz  profiles  are  more  highly  struc¬ 
tured  and  less  regular  than  the  SXR  time-intensity  curves, 
the  radio  time-integration  was  cut  off  strictly  at  10  min 
after  the  peak  of  the  SXR  burst.  After  some  experimenta¬ 
tion,  we  opted  to  begin  the  radio  integration  10  min  prior 
to  the  start  of  the  X-ray  integration.  The  units  of  the  radio 
integrated  intensity  are  sfu  minutes.  If  a  complete  or 


significant  data  gap  occurred  during  the  time  of  interest 
in  a  given  event,  the  integrated  radio  intensity  is  given  as 
Gap  in  Table  1  and  Data  Set  SI.  Other  events  occurred 
during  routine  calibration  intervals,  in  these  cases  "Cal"  is 
written  in  Table  1  and  Data  Set  SI.  The  Gap  and  Cal 
events  were  not  used  in  the  analysis  described  in  section 
3.3. 

3.2.  Individual  Event  Plots 
[12 J  Figures  4-6  contain  time-intensity  profiles  for  the 
1  -  8  A  SXR  and  1  MHz  radio  emissions  as  wrell  as  for  the 
>10  MeV  protons  for  three  representative  events  in  Data 


Table  2.  Soft  X-Ray  Flare  Integration  Flag 


SXR 

Fl< 

lio/lp 

Range 

Value 
Assigned 
to  110/1,» 

Decay  Time 
to  l,0/li>  < 
0.33  (min) 

Number 

of 

Cases1* 

1 

Hi)/ Ip  t  E0 

l,o/li*  *  0.75 

38 

15 

2 

0.85  <  ljo/lp  <  0.90 

1  „,/!,»  =  0.85 

68 

35 

3 

0.90  <  I,<)/1,»  :  0.95 

Win  =  0.90 

104 

3.3 

4 

0.95  <  Iin/I,>  <  1.0 

1«j/1i»  =  0.95 

214 

17 

5 

0.33  <  lio/lp  0.85 

No  Adjustment 

367 

6 

1,,,/lr  <  0.33 

No  Adjustment 

6 

7 

l,o(d)/Ip(d)  <  0.33 

No  Extrapolation 

228 

JAn  asterisk  (plus  sign)  following  the  numerical  flag  in  Table  1  and 
Data  Set  Si  indicates  that  the  rise  (decay)  time  of  the  SXR  burst  used 
for  the  integration  was  cut  off  at  60  (300)  min. 
bNo  SXR  data  for  three  events. 
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1-8 A  Peak  Date/Time;  08/23  09  34  e-folding  time:  00:15  X-ray  integration  interval:  09:31  -  09:44 

Class:  M  2.3  1/10  Max  Duration:  00:52  Waves  integration  interval:  09:21  -  09:44 

Location:  N32E33 

Figure  4.  (top)  Example  of  a  SXR  event  which  required  no  adjustment  to  the  slope  of  the 
extrapolated  exponential  fit  used  for  the  soft  X-ray  fluence  integration  between  the  1/3  power 
points.  The  red-hatched  area  indicates  the  SXR  fluence  based  on  the  extrapolated  fit  to  the  decay  of 
the  SXR  burst;  the  black-hatched  area  indicates  the  measured  SXR  fluence.  (middle  and  bottom) 
The  1  MHz  radio  (with  integration  indicated  by  black  hatching)  and  >10  MeV  proton  time-intensity 
profiles  (showing  data  from  all  available  GOES  satellites),  respectively. 


Set  SI.  Figure  4  shows  a  SXR  event  which  required  no 
adjustment  to  the  slope  of  the  extrapolated  exponential  fit 
used  for  the  soft  X-ray  integration  between  the  1/3  power 
points.  Figure  5  shows  an  event  for  which  l10/lp  =  0.988 
was  reset  to  0.95  in  accordance  with  our  prescription. 
Figure  6  shows  a  typical  "slow'  rise"  event  for  which  the 
soft  X-ray  integration  was  arbitrarily  initiated  60  min  prior 
to  the  event,  rather  than  at  the  1/3  power  point.  YVhile 
"well-behaved"  events  requiring  either  no  extrapolation 
or  no  adjustment  to  an  extrapolation  to  determine  the  SXR 
fluence  accounted  for  86%  (602/701)  of  the  sample,  we 
note  that  the  corresponding  percentage  for  SEP  associated 
flares  was  35%  (26/75),  reflecting  the  tendency  of  large  SEP 
events  to  be  associated  with  long-duration  flares.  Plots  in 
the  format  of  Figures  4  —  6  are  given  in  the  auxiliary 
material  for  all  of  the  704  events  in  Data  Set  Si  (Figures 
SI— S352)  as  well  as  for  all  SWPC  SEP  events  in  Table  1 
(Figures  S353-S399). 


3.3.  Methodology 

[33]  The  analysis  technique  known  as  logistic  regression 
[McCullagh  arid  N elder,  1983)  (first  applied  to  SEP  forecasts 
by  Garcia  (1994a))  can  be  used  to  obtain  a  continuous 
function  for  the  probability  of  SEP  event  occurrence  as  a 
function  of  the  integrated  radio  and  X-ray  flux  (as  com¬ 
puted  in  sections  3.1.2  and  3.1.3,  respectively)  for  the  SXR 
flares  in  our  sample.  This  technique  is  appropriate  for 
observational  studies  where  each  event  is  characterized  by 
binary  result  (yes  or  no)  and  associated  with  a  set  of 
explanatory  variables.  In  logistic  regression  analysis,  the 
dependent  variable  y  is  the  probability  (P)  that  an  event 
will  occur,  hence  it  is  constrained  between  0  and  1.  The 
logistic  model  is  written  as 


ll)8 ! 


Prob  (SEPevent 


[Pr  ob(  no_SEPt*vc*nt 


r  p  1  r 
loK  rh,  V  W 


’U  -PJ 


1  I 
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Figure  5.  (top)  Example  of  a  SXR  flare  for  which  the  measured  I10/Ii>  =  0.988  was  reset  to  0.95  in 
accordance  with  our  prescription.  The  red-hatched  area  indicates  the  SXR  fluence  based  on  the 
adjusted  fit  to  the  decay  of  the  SXR  burst;  the  black-hatched  area  indicates  the  measured  SXR 
fluence.  (middle  and  bottom)  The  1  MHz  radio  (with  integration  indicated  by  hatching)  and 
>10  MeV  proton  time-intensity  profiles,  respectively. 


where  //  =  *s  a  function  of  a  set  of  environmental 

/- i 

variables  xj  through  the  dj  coefficients.  Setting  these 
variables  to  be  the  logarithmic  value  of  the  integrated 
X-ray  flux  X  and  the  integrated  radio  flux  R,  the 
functional  relationship  between  the  probability  of  the 
event  P  and  //  becomes 

P(logX.logR)  — : —  (2) 

In  this  model  we  actually  have  two  more  terms  besides  the 
linear  log  X  and  log  R  components:  a  constant  term  and  an 
interaction  term  between  X  and  R.  We  did  not  include  a 
term  to  take  into  account  the  dependence  of  SEP  incidence 
on  heliographic  longitude,  but  we  computed  probability 
levels  separately  for  well-connected  (W20°~ W120°) 
events  as  well  as  for  two  longitude  ranges  of  increasingly 
poorer  connection  (E40°-VV19°  and  E120°-E41°).  The 
data,  consisting  of  binaiy  responses  spanning  the  log  X 


and  log  R  space,  yielded  the  following  model  for  each  of 
the  three  considered  longitude  intervals,  respectively, 

=  -6  07  -  1.75  Iog1()  (X)  +  1  14  logjQ  (R) 

+  0.56  log J0  (X)  log10  (R). 

,/2  =  -7.44  -  2.99 log  10  (X)  f  1.21  log ]0{R)  (3) 

0*69  Iogl0  (X)  Iog10  (R). 
lh  5.02  -  1  74 1°8io  W  +  0  64  lo8iu  iR) 

*e  0.40loglu  (X)  logH)  (R). 

The  goodness  of  the  method  was  estimated  by  computing 
the  standard  deviation  of  the  difference  between  the 
continuous  function  P(log  X,  log  R)  and  the  binary 
occurrence  of  SEPs  in  the  sample  considered.  This 
estimated  standard  deviation  is  a  =  0.2.  Although  this  is 
relatively  high,  the  method  yields  reasonable  results. 

[34]  The  results  of  the  model  are  presented  in  Figures  7-9 
which  contain  seatterplots  of  the  time-integrated  1  MHz 


10  of  18 


S04008 


LAURENZA  ET  AL.:  WARNINGS  OF  SEP  EVENTS 


S04008 


1-8A  Peak  Date/Time:  11/01  15:18  e-folding  time:  01:19  X-ray  Integration  Interval:  14:18  - 15:28 

Class:  M  2.0  1/10  Max  Duration:  07:01  Waves  integration  interval:  14  08  -  15:28 

Location.  S18E79 


Figure  6.  (top)  Example  of  a  slow  rise  SXR  flare  for  which  the  soft  X-ray  integration  (indicated  by 
the  hatched  area)  was  arbitrarily  initiated  1  hour  prior  to  the  event  peak  and  for  which  the 
measured  Ijo/Ip  =  0.923  was  reset  to  0.90  in  accordance  with  our  prescription.  The  red-hatched  area 
indicates  the  SXR  fluence  based  on  the  adjusted  fit  to  the  decay  of  the  SXR  burst;  the  black-hatched 
area  indicates  the  measured  SXR  fluence.  (middle  and  bottom)  The  1  MHz  radio  (with  integration 
indicated  by  hatching)  and  >10  MeV  proton  time-intensity  profiles,  respectively. 


radio  flux  versus  the  time-integrated  SXR  flux,  with  con¬ 
tours  for  SEP  event  probability  computed  from  equations 
(2)  and  (3),  for  events  in  Data  Set  SI  that  fell  in  the  longitude 
ranges  of  good,  intermediate,  and  poor  connection,  respec¬ 
tively.  The  E40°  and  W20°  borders  between  these  three 
heliolongitude  bins  were  determined  by  trial  and  error  to 
optimize  the  forecast  technique  performance.  For  all  three 
longitude  ranges  it  can  be  seen  that  the  probability  predic¬ 
tion  functions  reflect  SEP  event  occurrence  dependence  on 
the  time-integrated  X-ray  and  radio  fluxes,  although  a 
number  of  SEP  (non-SEP)  events  are  located  in  the  lovv- 
(high-)  probability'  zone.  In  principle,  every  solar  flare  can 
be  located  in  such  diagrams,  given  the  key  parameter 
values,  in  order  to  evaluate  the  probability  of  a  subsequent 
SEP  event.  The  method  allows,  when  choosing  a  specific 
probability  contour,  a  yes/no  binary  response  for  the 
occurrence  of  a  SEP  event  as  follows:  if  flare  parameters 
are  located  above  the  selected  probability  curve,  a  warning 
is  issued  otherwise  no  alert  is  given,  as  will  be  extensively 
described  in  the  next  section. 


3.4.  Evaluation  of  the  Model 
3.4.1.  Accuracy 

[35 1  The  performance  of  the  obtained  probability  fore¬ 
casts  can  be  evaluated,  covering  the  data  set  on  which  it 
was  developed,  in  terms  of  False  Alarm  Rate  (FAR)  and 
Probability  of  Detection  (POD)  as  tollows.  Let  us  define  a 
probability  threshold  p,  (that  corresponds  to  selecting  a 
contour  line  on  plots  7  —  9)  such  that  a  warning  will  be 
issued  whenever  the  forecast  probability  (fp)  satisfies  the 
condition  fp  >  pt  and  no  warning  will  be  issued  if  fp  <  p,. 
Then  the  forecasts  and  observations  can  be  compared  in 
terms  of  the  following  variables:  the  number  of  correct 
forecasts  or  hits,  A  (an  SVVPC  SEP  event  was  forecast  and 
one  occurred);  the  number  of  false  alarms,  B  (an  SVVPC 
SEP  event  was  forecast  but  none  occurred);  the  number  of 
missed  events,  C  (no  SVVPC  SEP  event  was  predicted  blit 
an  event  did  occur);  the  number  of  correct  nulls,  D  (no 
SVVPC  SEP  event  forecast  and  none  occurred);  the  number 
of  forecasts  we  would  expect  to  be  correct  by  chance,  E; 
the  total  number  of  forecasts  (both  positive  and  null),  N. 


11  Dl  18 


S04008 


LAURENZA  ET  AL.:  WARNINGS  OF  SEP  EVENTS 


S04008 


C 


Figure  7.  Integrated  1  MHz  radio  intensity  versus  integrated  1-8  A  soft  X-ray  intensity  for  >M2 
soft  X-ray  flares  from  1995  to  2005  located  from  W20°-W120°.  Solid  circles  represent  flares 
associated  with  SWPC  SEP  events.  The  black  thick  contour  indicates  the  probability  threshold 
used  in  the  analysis. 


The  following  statistical  quantities  specify  the  quality  of 
these  categorical  forecasts:  (1)  probability  of  detection 
POD  =  A/(A  +  C);  (2)  false  alarm  rate  FAR  =  B/(A  +  B); 
(3)  percent  correct  PC  =  (A  +  D)/N;  and  (4)  Heidke  Skill 
Score  HSS  =  (A  +  D  -  E)/(N  E),  that  is  the  fraction  of 

correct  forecasts  adjusted  by  E  =  [(A  +  B)(A  +  C)  +  (B  + 
D)(C  +  D)|/N  \Balch,  2008],  i.e.,  the  number  of  correct 
forecasts  by  chance,  that  can  derived  according  to  the 
following  argument.  Given  that  Probability  (event  =  Yes)  = 
(A  +  C)/N  and  Probability  (forecast  =  Yes)  =  (A  +  B)/N, 
the  probability  for  a  chance  hit  is  the  combined  probability 
(A  +  C)(A  +  B)/N2.  Moreover,  the  probability  of  a  chance 
correct  null  is  the  product  of  Probability  (event  =  No)  =  (B  + 
D)/N  and  Probability  (forecast  =  No)  =  (C  +  D)/N;  hence 
Probability  (event  =  No  and  forecast  =  No)  =  (B  +  D)(C  +  D)/ 
N2.  The  combined  probability  for  a  chance  correct  forecast 
(hits  and  correct  nulls)  is  [(A  +  C)(A  +  B)  +  (B  +  D)(C  +  13)]/ 
N“.  We  therefore  derive  the  number  of  correct  forecasts  by 
chance  to  be  E  =  [(A  +  C)(A  +  B)  +  (B  +  D)(C  +  D)J/N. 


[36]  For  probability  forecasts,  the  probability  threshold 
can  be  treated  as  an  independent  variable  ranging  from 
0.0  to  1.0,  hence  each  of  the  categorical  quality  measures 
(POD,  FAR,  PC,  HSS)  can  be  considered  to  be  a  function  of 
pt.  Figure  10  displays  the  categorical  quality  measures  for 
the  current  method  versus  the  probability  threshold  level: 
POD  (thick  line)  and  FAR  (thin  line)  trends  are  shown  in 
the  Figures  10a -10c  for  the  three  considered  longitude 
intervals;  HSS  (thick  line)  and  PC  (thin  line)  are  displayed 
in  Figures  1  Od  —  1  Of  as  well.  Note  that  both  the  POD  and 
the  FAR  are  quite  high  for  lower-probability  thresholds 
and  decrease  w  ith  increasing  threshold;  HSS  optimization 
is  generally  achieved  for  range  of  probabilities  20-40%. 
The  optimal  choice  for  p<  involves  a  trade  off  between 
maximizing  the  POD  and  minimizing  the  FAR,  while 
achieving  the  HSS  optimization.  For  the  western  events 
(Figure  1  Of)  the  optimal  point  is  p,  =  28%  (POD  =  bl  %  (29/ 
47),  FAR  =  41%  (20/49),  1 1SS  =  0.53);  for  the  "intermediate" 
events  (Figure  lOe),  the  optimal  is  reached  for  p,  =  28% 
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I  igure  8.  Integrated  1  MHz  radio  intensity  versus  integrated  1-8  A  soft  X-ray  intensity  for  >M2 
soft  X-ray  flares  from  1995  to  2005  located  from  E40°-W19°.  Solid  circles  represent  flares 
associated  with  SVVPC  SEP  events.  The  black  thick  contour  indicates  the  probability  threshold 
used  in  the  analysis. 


(POD  =  88%  (15/22),  FAR  =  46%  (13/28),  HSS  =  0.60);  and 
for  the  eastern  events  (Figure  10d),  the  optimal  p,  =  30% 
(POD  =  50%  (3/6),  FAR  =  25%  (1/4),  HSS  =  0.59). 

[37[  In  calculating  these  verification  measures,  we  took 
into  account  that  12  SEP  events  were  associated  with  disk 
flares  of  class  <M2;  9  of  these  12  "Misses"  (since  they 
cannot  be  predicted  by  our  method)  fell  in  the  western 
longitude  bin  and  3  in  the  central  bin.  Also,  we  note  that 
our  extrapolations/adjustments  to  the  SXR  flux  resulted  in 
a  gain  of  five  hits  (caused  by  events  for  which  used 
tluence/measured  fluence  in  Data  Set  SI  was  >1.0  (Nos. 
23,  435,  446,  618,  664)),  a  loss  of  one  hit  (used  fluence/ 
measured  fluence  <1 .0  (No.  408)),  a  gain  of  two  False  Alarms 
(caused  by  events  for  which  used  tluence/measured  flu- 
ence  in  Data  Set  SI  was  >1.0  (Nos.  280,  484)),  and  a  loss  of 
two  False  Alarms  (caused  by  events  for  which  used  flu- 
ence/mensured  fluence  in  Data  Set  SI  was  <1.0  (Nos.  519, 


566)).  For  the  above  cases  the  ratio  of  the  used  to  the 
measured  SXR  fluence  ranged  from  0.355  to  3.53. 

[3hJ  Table  3  show  the  2x2  contingency  matrix  with 
values  of  A,  B,  C,  and  D,  obtained  by  combining  events 
from  all  the  heliolongitude  ranges.  The  combined  result 
for  all  longitudes  is  then:  POD  =  63%  (47/75),  FAR  =  42% 
(34/81),  HSS  =  0.58  and  PC  =  93%  (633/683).  While  this 
result  is  a  clear  improvement  over  that  given  by  the 
current  Protons  model  used  at  SVVPC  (POD  =  57%,  FAR 
=  55%  IBalch,  1999,  2008[)  and  the  2-D  probability  models 
(best  performer:  KISS  =  0.55,  POD  =  54%,  FAR  =  42% 
[Btilch,  2006])),  it  falls  well  short  of  that  obtained  with 
forecaster  input  (POD  =  88%;  FAR  =  18%). 

3.4.2.  Warning  Time 

[39]  One  of  our  guiding  principles  was  the  requirement 
to  maximize  the  interval  from  the  time  that  a  warning  is 
issued  until  the  SEP  event  onset.  Normally,  the  SEP  onset 
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Figure  9.  Integrated  1  MHz  radio  intensity  versus  integrated  1-8  A  soft  X-ray  intensity  for  >M2 
soft  X-ray  flares  from  1995  to  2005  located  at  longitudes  from  E120°-E41°.  Solid  circles  represent 
flares  associated  with  SVVPC  SEP  events.  The  black  thick  contour  indicates  the  probability 
threshold  used  in  the  analysis. 


would  correspond  to  the  end  of  the  third  5-min  interval  with 
J  (>10  MeV)  >10  pfu).  1  lowever,  as  Posner  [2007]  has  recently 
shown,  the  GOES  >10  MeV  SEP  time  profiles  are  affected  by 
contamination  from  relativistic  electrons/protons  during 
the  SEP  onset  period.  Thus,  we  did  not  use  the  GOES 
>10  MeV  SEP  data  to  determine  onset  times  for  the  47 
successful  forecasts  (//hits,/)  made  (post  hoc)  by  our  tech¬ 
nique  for  solar  cycle  23,  but  instead  used  the  31  -50  MeV 
proton  onset  times  published  by  Posner  (2007)  for  19  of 
these  47  events.  For  the  19  common  events,  we  obtain 
delay  times  ranging  from  7  min  to  882  min  with  a 
median  of  39  min.  For  comparison,  for  these  events  Posner 
obtained  a  range  from  13  min  to  727  min  (median  =  35  min). 
Note  that  the  onset  times  obtained  by  Posner  refer  not  to 
the  NOAA  event  threshold  crossing  (or  equivalent  at  3 1  — 
50  MeV)  but  rather  to  the  rise  of  the  event  above  the 
preevent  background.  Even  for  very  fast  rise  events,  the 
required  15-min  interval  above  threshold  to  confirm  a 
NOAA  SEP  event  indicates  that  advance  warning  (with  a 


range  of  8  min  to  897  min  and  a  median  of  54  min)  would 
have  been  issued  for  all  19  events  by  our  technique. 

4.  Discussion 

4.1.  Recently  Developed  SEP  Warning 
Techniques 

(40 J  The  current  emphasis  on  solar  system  exploration 
and  the  attendant  radiation  threat  to  astronauts  have 
resulted  in  increased  emphasis  on  SEP  event  forecasting. 
With  "advanced  (physics-based)  predictive  models"  lying 
"a  number  of  years  in  the  future"  [Baker  et  al.,  2007), 
empirical  models  or  decision  aids  such  as  that  presented 
here  will  have  to  bridge  the  gap.  Several  articles  [Kubo  and 
Akioka,  2004;  Garcia,  2004a,  2004b;  Posner,  2007;  Batch,  2008) 
have  focused  on  such  techniques.  How  do  the  feasibility 
and  performance  of  the  method  presented  here  compare 
with  that  of  the  techniques  presented  in  these  recent 
articles? 
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Figure  10.  SEP  event  forecast  performance  based  on  probability  thresholds.  POD  (thick  line)  and 
FAR  (thin  line)  as  functions  of  the  probability  threshold  for  (a)  eastern,  (b)  intermediate,  and 

(c)  western  events.  HSS  (thick  line)  and  PC  (thin  line)  as  functions  of  the  probability  threshold  for 

(d)  eastern,  (e)  intermediate,  and  (f)  western  events. 


fill  1.  Knbo  find  Akioka  [2004]  showed  that  tor  both  solar 
cycles  22  and  23,  an  integrated  SXR  flux  of  —20  ergs  cm  2 
was  an  almost  necessary  condition  (POD  —100%)  for  a 
disk  Hare  to  be  followed  by  an  SWPC  SEP  event.  They 
note  that  while  400  SXR  flares  had  time-integrated  fluxes 
of  —20  ergs  cm  2  since  1997,  only  60  SWPC  SEP  events 
occurred  during  this  time,  indicating  a  high  (—85%)  false 
alarm  rate  if  this  method  was  used  by  itself. 

|42]  2.  G<mm  [2004a]  used  a  combination  of  SXR  flare 
peak  intensity  and  peak  flare  temperature  (given  by  the 
ratio  ot  the  0.5-3  A  SXR  flux  to  the  1-8  A  tlux  {Garcia, 
1994b]).  For  the  optimum  method  based  on  these  two 
parameters,  and  additional  constraints  based  on  flare 
duration  and  flare  location,  the  reported  POD  was  94% 
(46/49)  and  the  FAR  was  46%  (39/85).  If,  however,  w'e 
calculate  the  POD  as  was  done  for  our  technique,  i.e., 
taking  into  account  all  SWPC  SEP  events  associated  with 
disk  SXR  flares  of  any  size,  then  the  POD  drops  to  58% 
(46/80). 

[43]  3.  Garcia  [2004b]  used  the  hard  X-ray  spectral  index 
(as  determined  by  the  NOAA/Czech  I1XRS  experiment 
{Fiiniik  et  a!.,  2001|)  as  an  input  parameter  for  SEP  predic¬ 
tion  and  obtained  a  POD  of  88%  ( 14/16)  and  a  FAR  of  18% 
(3/17)  from  March  2000  through  December  2002.  However, 
SWPC  SEP  events  associated  with  disk  flares  too  weak 
(  Ml. 9)  to  permit  spectral  analysis  were  not  considered  as 
missed  events  (nine  such  SEP  events  occurred  during  the 
interval  used,  see  Table  1)  and  1 1  SXR  flares  (many  with 


the  flat  spectral  signature  associated  with  SEP  events)  that 
occurred  when  a  SEP  event  was  still  in  progress  were  not 
considered  in  the  analysis.  Both  of  these  effects  will  degrade 
the  performance  of  the  Garcia  [2004b]  hard  X-ray  tech¬ 
nique.  That  said,  Kiplittger  [1995]  reported  comparable 
results  (POD  =  96%  (22/23)  and  FAR  =  27%  (8/30))  for  a 
similar  prediction  method  based  on  hard  X-ray  spectra, 
indicating  promise  for  this  approach. 

[44]  4.  Most  recently,  Posner  [2007]  explored  the  use  of 
relativistic  electrons  as  precursors  for  >30  MeV  proton 
events  (see  Kuwabara  et  id.  [2006)  for  a  ground  level  event 
(>500  MeV  protons)  alarm  system  based  on  neutron 
monitor  data).  The  method  was  developed  for  1 996  — 
2002  and  tested  for  year  2003,  during  which  it  predicted 
4  of  5  SEP  events  (POD  =  80%)  with  a  FAR  of  56%  (5/9), 
although,  as  discussed  by  the  author,  there  are  mitigating 
factors  for  several  of  the  false  alarms.  For  the  year  2003, 


Table  3.  Contingency  Matrix  for  Evaluating  the  Forecast 
Model*' 


Event  Observed:  Yes 

Event  Observed:  No 

Event  Forecast:  Yes 

A  =  47 

B  =  34 

Event  Forecast:  No 

C  =  16  (+12) 

D  =  586 

*The  numbers  (excluding  the  12  SEP  events  associated  with  <M2 
flares)  do  not  add  to  704  because  of  missing  SXR  or  radio  data  for  21 
events. 


15  of  18 


S04008 


LAURENZA  ET  AL.:  WARNINGS  OF  SEP  EVENTS 


S04008 


our  method  (evaluated  as  above  for  SWPC  threshold 
events)  yielded  comparable  results,  with  a  POD  of  87% 
(7/8)  and  a  FAR  of  46%  (6/13).  The  difference  in  the  number 
of  threshold  SEP  events  (5  for  Posner  |2007|  versus  10  in  our 
Table  1)  is  due  to  Posner  [2007]  use  of  the  30-50  MeV 
channel  on  the  Comprehensive  Suprathermal  and  Energetic 
particle  Analyzer  (COSTEP)  instrument  on  SOHO  rather 
than  the  GOES  >10  MeV  channel  and  a  higher  event 
trigger  intensity  threshold  (--20  pfu  versus  10  pfu). 
Because  of  the  different  SEP  database  on  which  the  Posner 
[2007]  technique  was  developed,  a  direct  comparison  of  it 
with  our  method  is  not  possible;  we  can  state  that  the  two 
methods  give  comparable  results  for  the  SEP  energies  and 
event  thresholds  for  which  they  were  developed. 

4.2.  Standardization 

[ 45 1  The  above  revievv/comparison  highlights  the  need 
for  standardized  databases  and  definitions  for  SEP  warn¬ 
ing  studies.  Standardized  comparisons  are  required  if 
space  weather  forecasting  is  to  follow  the  path  of  steady 
incremental  improvement  that  has  characterized  progress 
in  terrestrial  weather  forecasting  [Siscoe,  20061. 

1 4<> J  Our  first  suggestion  is  to  use  a  standardized  data¬ 
base  for  testing  new  algorithms.  Toward  this  end,  we  have 
included  both  our  SEP  event  and  SXR  data  (in  the  auxil¬ 
iary  material)  bases,  as  Table  1  and  Data  Set  S'l,  respec¬ 
tively.  Thus  anyone  washing  to  improve  on  our  technique 
can  compare  results  based  on  the  same  sample  of  events, 
in  order  to  verify  gains. 

[47]  Our  second  suggestion  is  standardizing  protocols 
for  identifying  hits,  misses,  and  false  alarms.  For  example, 
in  this  study: 

|48]  1.  We  used  a  "one  SEP  event,  one  flare"  association 
rule,  with  the  credited  solar  event  being  the  one  that  took 
the  SEP  event  across  the  >10  pfu  threshold.  This  is 
necessary  because  a  "yes/no"  SWPC  SEP  event  decision 
needs  to  be  made  for  each  "triggering"  flare  (i.e.,  >M2)  in 
our  scheme. 

[49 ]  2.  We  issued  a  hypothetical  forecast  for  every  soft 
X-ray  event  that  met  our  >M2  threshold,  regardless  of 
flare  location  (frontside/backside,  observed/inferred)  and 
the  SEP  background  intensity  at  the  time  of  the  flare.  If  no 
Ho  flare  was  reported  in  association  with  a  >M2  flare,  we 
made  our  best  guess  as  to  the  origin  based,  e.g.,  on  active 
region  flare  histories,  and  issued  a  (post  hoc)  yes/no 
prediction,  because  this  is  what  the  duty  forecaster  would 
need  to  do.  For  a  particle  based  technique  such  as  that  of 
Posner  [2007],  the  triggering  event  would  not  be  a  flare  but 
an  electron  (or  high-energy  proton  event)  above  a  certain 
threshold. 

1 50 1  3.  We  required  an  increase  of  a  factor  of  two  to 
register  a  hit  when  the  triggering  >M2  flare  occurred 
when  the  SEP  intensity  was  already  above  the  SWPC 
event  threshold.  If  such  an  increase  was  not  achieved 
following  a  positive  forecast,  a  false  alarm  was  registered. 
This  approach  differs  from  that  used  by  SWPC  (K.  Doggett, 
personal  communication,  2007).  If  the  SEP  intensity  at  the 


time  of  a  new'  (triggering)  flare  is  already  above  threshold, 
SWPC  will  not  issue  a  new  (positive  or  negative)  forecast, 
although  the  end  time  of  the  event  in  progress  may  be 
extended.  Our  procedure  will  yield  a  greater  number  of 
hits  and  false  alarms. 

1 5i ]  4.  We  counted  all  SEP  events  associated  with  <M2 
disk  flares  as  misses  (even  though  no  forecast  was 
attempted). 

(52|  5.  We  ignored  small  (<M2)  behind  the  limb  events 
for  which  the  input  data  may  have  been  compromised. 
This  limitation  may  not  apply  to  all  techniques,  e.g.,  the 
electron -based  technique  of  Posner  [2007 ] .  In  such  cases, 
techniques  should  be  evaluated  on  an  equal  footing  with 
special  events  such  as  these  treated  the  same  in  the 
evaluation  of  all  methods  being  compared. 

[53]  Our  final  suggestion  is  that  new  forecast  methods 
should  determine  the  delay  from  the  time  of  forecast  to 
the  >10  MeV  onset  (data  permitting)  for  comparison  with 
existing  techniques  (see  the  discussions  of  warning  lead 
times  given  by  Garcia  |2004a]  and  Posner  [2007]). 

4.3.  Current  Objective  Techniques  for  SEP  Event 
Forecasting:  Moving  Beyond  50-50 

[54]  Oliver  et  al.  [1985]  reported  that  the  U-shaped 
microwave  burst  criterion  for  predicting  SWPC  SEP 
events  | Castelli  et  al.,  1967]  had  both  POD  and  FAR  of 
—50%  for  the  1965-1979  time  interval.  When  one  consid¬ 
ers  that  by  raising  and  lowering  the  SXR  prediction 
threshold  in  a  given  technique  (say  from  Cl  to  X10  in 
our  scheme),  one  can  force  results  ranging  from  POD 
-100%  and  FAR  -100%  to  POD  -0%  and  FAR  -0%,  50/50 
represents  the  baseline  starting  point  for  progress.  The 
technique  presented  in  this  study  yielded  a  POD  of  63% 
and  a  FAR  of  42%  (for  the  years  on  which  it  was  devel¬ 
oped).  These  results,  as  well  as  all  those  for  the  other 
recent  studies  discussed  above  (with  the  possible  excep¬ 
tion  of  the  hard  X-ray  technique  of  Garcia  [2004b]),  pale  in 
comparison  with  the  current,  forecaster-in-the-loop, 
method  currently  in  use  at  SWPC. 

[55 1  The  reason  for  this  is  that  the  forecaster  brings  both 
experience  and  more  observations  to  the  task.  Note  that  all 
of  the  above  techniques  involve  at  the  most  three  variables 
including,  generally,  a  measure  of  flare  size  and  location. 
Generally  speaking,  the  more  data  brought  to  bear  on  a 
forecast,  the  better  the  result.  For  example,  a  simple  two- 
dimensional  plot  of  integrated  SXR  intensity  (from  the 
SXR  fluence  used  information  in  Data  Set  SI)  versus  flare 
longitude  yields  a  FAR  of  64%  for  a  POD  of  63%,  a  poorer 
result  than  that  obtained  when  the  third  input  parameter 
(integrated  radio  flux)  was  considered.  None  of  the  tech¬ 
niques  discussed  here  takes  into  account  the  SEP  history 
of  a  given  region.  Given  the  tendency  for  SEP  events  to 
occur  in  clusters  [e.g.,  Svestka,  1968;  Oliver,  198U],  such 
knowledge  will  figure  prominently  in  the  forecaster's 
evaluation  of  a  flare  which  exceeds  the  SEP  forecast 
trigger  threshold.  Other  factors  such  as  the  SEP  back¬ 
ground  flux  near  Earth  at  the  time  of  a  flare  may  also  play 
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o  role  [Cliver,  2006;  Mewaldt  et  nL,  2006].  And  combined 
methods  using  both  flare  electromagnetic  and  particle 
input  also  offer  promise,  particularly  for  SEP  events  orig¬ 
inating  from  behind-the-limb  flares  where  occupation 
effects  compromise  electromagnetic  emissions.  Until  the 
performance  of  objective  methods,  either  empirical  or  phys¬ 
ics-based,  exceeds  those  of  the  "forecaster-in-the-loop" 
model,  however,  the  use  of  such  techniques  is  necessarily 
relegated  to  that  of  a  decision  aid. 
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